new driver: kubernetes #167
Conversation
Would it make sense that if you do |
|
That doesn't seem to work well with Deployment lifecycle |
|
@AkihiroSuda Why? They would all be running or inactive based on if the deployment is running or not. Can't we even have it so that the consistent hashing (optionally) works if you just add two container nodes manually? |
|
You need to synchronize buildx state with Kube Deployment when pods are rescaled (by either user or HPA) or recreated. Also, it seems hard to bind pod lifecycle to buildx node without manually managing pods (i.e. reinventing Deployment) or using StatefulSet which is slow. |
|
Can we rename buildx "Nodes" to "Instances"? |
|
Or just add a document to clarify buildx "Node" is unrelated to Kubernetes "Node". |
AkihiroSuda
force-pushed
the
kube
branch
2 times, most recently
from
9d04f5f
to
e3079f0
Compare
driver/bkimage/bkimage.go
Outdated
| package bkimage | ||
|
|
||
| const ( | ||
| DefaultImage = "moby/buildkit:buildx-stable-1" // TODO: make this verified |
There was a problem hiding this comment.
@tonistiigi Is this different from moby/buildkit:v0.6.2?
There was a problem hiding this comment.
It is atm, but it is a different tag in case we need to break compatibility in the future. Eg. moby/buildkit:v0.7.0 might be equal to moby/buildkit:buildx-stable-1 or moby/buildkit:buildx-stable-2 depending if it is compatible.
AkihiroSuda
force-pushed
the
kube
branch
2 times, most recently
from
99d6dc4
to
c7d30e9
Compare
|
Ready for review |
Instance is what we call one named builder already. Instance can have many nodes. |
|
Subinstance? |
|
How can we move this forward? |
|
added new (hacky) commit: kubernetes: show Kubernetes Pods as buildx "Nodes" in $ k get pods
NAME READY STATUS RESTARTS AGE
kube0-75897b8444-dnfhg 1/1 Running 0 2m
kube0-75897b8444-dsfht 1/1 Running 0 2m
kube0-75897b8444-ghthr 1/1 Running 0 2m
$ docker buildx inspect
Name: kube
Driver: kubernetes
Nodes:
Name: kube0-75897b8444-dnfhg
Endpoint:
Status: running
Platforms: linux/amd64, linux/386
Name: kube0-75897b8444-dsfht
Endpoint:
Status: running
Platforms: linux/amd64, linux/386
Name: kube0-75897b8444-ghthr
Endpoint:
Status: running
Platforms: linux/amd64, linux/386 |
Tested with `kind` and GKE. Note: "nodes" shown in `docker buildx ls` are unrelated to Kubernetes "nodes". Probably buildx should come up with an alternative term. Usage: $ kind create cluster $ export KUBECONFIG="$(kind get kubeconfig-path --name="kind")" $ docker buildx create --driver kubernetes --driver-opt replicas=3 --use $ docker buildx build -t foo --load . `--load` loads the image into the local Docker. Driver opts: - `image=IMAGE` - Sets the container image to be used for running buildkit. - `namespace=NS` - Sets the Kubernetes namespace. Defaults to the current namespace. - `replicas=N` - Sets the number of `Pod` replicas. Defaults to 1. - `rootless=(true|false)` - Run the container as a non-root user without `securityContext.privileged`. Defaults to false. - `loadbalance=(sticky|random)` - Load-balancing strategy. If set to "sticky", the pod is chosen using the hash of the context path. Defaults to "sticky" Signed-off-by: Akihiro Suda <[email protected]>
…inspect` Signed-off-by: Akihiro Suda <[email protected]>
|
I'm merging, we can improve in followups. |
Tested with
kindand GKETODOs:
No Auth Provider found for name "gcp"error on GKEkubectlbinarydocker contextkubetokubernetesfordockerCLI consistencyNote: "nodes" shown in
docker buildx lsare unrelated to Kubernetes "nodes".Probably buildx should come up with an alternative term.
Usage:
--loadloads the image into the local Docker.Driver options
image=IMAGE- Sets the container image to be used for running buildkit.namespace=NS- Sets the Kubernetes namespace. Defaults to the current namespace.replicas=N- Sets the number ofPodreplicas. Defaults to 1.rootless=(true|false)- Run the container as a non-root user withoutsecurityContext.privileged. Using Ubuntu host kernel is recommended. Defaults to false.loadbalance=(sticky|random)- Load-balancing strategy. If set to "sticky", the pod is chosen using the hash of the context path. Defaults to "sticky"Signed-off-by: Akihiro Suda [email protected]
Close #22